Deciphering Severely Degraded License Plates

Extremely low-quality images, on the order of 20 pixels in width, appear with frustrating frequency in many forensic investigations. Even advanced de-noising and super-resolution technologies are unable to extract useful information from such lowquality images. We show, however, that useful information is present in such highly degraded images. We also show that convolutional neural networks can be trained to decipher the contents of highly degraded images of license plates, and that these networks significantly outperform human observers. Introduction Recognizing text in images is a well-studied problem [1]. Text recognition can either be done by recognizing individual characters or by recognizing the full word. For degraded images, however, it is difficult to localize and recognize individual characters in an image. Word recognition, therefore, has become central to text recognition in degraded images. Deep convolutional neural networks [2] have been used for text recognition in natural images. Goodfellow et al. [3] used a deep neural network to localize, segment and recognize multiple digits on street view images. Jaderberg et al. [4] also proposed an end to end text recognition system for natural images using a deep neural network. Jaderberg et al. used a large word dictionary and formulated the text recognition task as a large scale classification problem. Recently, Svoboda et al. [5] used CNN to remove motion blur from images of license plate blurred with a blur kernel of various directions and lengths. Although this approach is able to deblur highly blurred images, it does not contend with extremely low resolution and noisy images. Unlike much of this previous work we focus on extracting text from highly degraded images on the scale of only a few pixels per character. Hsieh et al. [6] were the first to show that information can be extracted from highly degraded license plates. In this work the authors assume a known font type, font size, and character layout, and assume that the degraded image is blurry and perspectively distorted, but does not necessarily contain additive noise. Although the authors only show results on a small set of images, they do show that information is present in license plates as small as 20 pixels in width. Building on these ideas, in this paper we propose to train a CNN for recognizing highly degraded license plates with an unknown background template, font type, size and character location. We also explicitly work in the presence of high amounts of additive noise – a common occurrence in real-world imagery. Recognition by Human Observers We begin by performing a perceptual study to determine how well human observers can decipher degraded license plates. This study provides a baseline against which to compare our computational approaches. Observers were shown images of synthetically Figure 1. An example of the type of degraded license plate that we seek to